Skip to main content

Run

A collection can be executed multiple times. A Run is a single execution of a collection.

Endpoints​

POST /v1/async/collections/{collection_id}/run​

This endpoint triggers a Run of a collection.

A collection_id is required to make this request.

Response Example:

{
  "run_id": "9b64941a-4545-4c57-9174-c70e781d9192",
  "status": "in_progress",
  "total_requests": 2,
  "success_requests": 0,
  "failed_requests": 0,
  "timeout_requests": 0,
  "collection_id": "9634997b-6431-4b11-a4cb-fc00e941ba8d",
  "job_ids": ["job-uuid-1", "job-uuid-2"],
  "callback_url": "https://your-server.com/webhook",
  "callback_status": "pending"
}

Details about the returned fields can be found in Reference.

GET /v1/async/collections/{collection_id}/runs/{run_id}​

This endpoint returns the current status of a Run, including the webhook delivery status.

Response Example

{
  "run_id": "9b64941a-4545-4c57-9174-c70e781d9192",
  "status": "completed",
  "total_requests": 2,
  "success_requests": 2,
  "failed_requests": 0,
  "timeout_requests": 0,
  "collection_id": "9634997b-6431-4b11-a4cb-fc00e941ba8d",
  "job_ids": ["job-uuid-1", "job-uuid-2"],
  "callback_url": "https://your-server.com/webhook",
  "callback_status": "sent"
}

GET /v1/async/collections/{collection_id}/runs/{run_id}/jobs/{job_id}/result​

Retrieves the individual result of a job. Results are available for 24 hours.

Response Example

{
  "html": "<!doctype html>...",
  "statusCode": 200,
  "timings": {"queue_wait_ms": 45, "proxy_ms": 120},
  "potentiallyBlockedByCaptcha": false
}

Webhooks​

If the collection has a callback_url configured, a signed HTTP POST is automatically sent upon run completion:

{
  "event": "run.completed",
  "run_id": "uuid",
  "collection_id": "uuid",
  "status": "completed",
  "total_requests": 2,
  "success_requests": 2,
  "failed_requests": 0,
  "job_ids": ["job-uuid-1", "job-uuid-2"],
  "results_url": "https://api.scrapingpros.com/v1/async/collections/{cid}/runs/{rid}",
  "timestamp": "2026-04-06T20:30:00Z"
}

Security: The webhook includes an HMAC-SHA256 signature in the headers:

  • X-SP-Signature: sha256=<hex> -- signature of {timestamp}.{body}
  • X-SP-Timestamp: <unix_epoch>

Retries: If delivery fails (timeout, 5xx), it is automatically retried up to 5 times with backoff: 1min, 5min, 30min, 2h, 12h. The callback_status field reflects the current status.

Reference​

  • run_id: Generated UUID of the run. This value is recommended for run tracking using GET /v1/async/collections/{collection_id}/runs/{run_id}.
  • status: The current status of the Run. It can take 2 values: in_progress or completed.
  • total_requests: Number of requests in the collection.
  • success_requests: Number of requests that succeeded.
  • failed_requests: Number of requests that failed.
  • timeout_requests: Number of requests that timed out.
  • collection_id: UUID of the collection.
  • job_ids: List of UUIDs of the individual jobs. Use these to retrieve results with the job result endpoint.
  • callback_url: Configured webhook URL (if set).
  • callback_status: Webhook delivery status: pending (in progress), sent (delivered), failed (delivery failed), retrying (retrying delivery).